Markov Processes with Identical Bridges

Markov Processes with Identical Bridges

Let X and Y be time-homogeneous Markov processes with common state space E, and assume that the transition kernels of X and Y admit densities with respect to suitable reference measures. We show that if there is a time t > 0 such that, for each x ∈ E, the conditional distribution of (Xs)0≤s≤t, given X0 = x = Xt, coincides with the conditional distribution of (Ys)0≤s≤t, given Y0 = x = Yt, then t...

ON THE INFINITE ORDER MARKOV PROCESSES

The notion of infinite order Markov process is introduced and the Markov property of the flow of information is established.

Markov Processes with Restart

We consider a general honest homogeneous continuous-time Markov process with restarts. The process is forced to restart from a given distribution at time moments generated by an independent Poisson process. The motivation to study such processes comes from modeling human and animal mobility patterns, restart processes in communication protocols, and from application of restarting random walks i...

The KTH Visit in Semi-Markov Processes

Markov Decision Processes with Arbitrary Reward Processes

We consider a learning problem where the decision maker interacts with a standard Markov decision process, with the exception that the reward functions vary arbitrarily over time. We show that, against every possible realization of the reward process, the agent can perform as well—in hindsight—as every stationary policy. This generalizes the classical no-regret result for repeated games. Specif...